Skip to content

ci: faster Test Core — affected-only PR tests, single spec suite run#2925

Merged
os-zhuang merged 1 commit into
mainfrom
claude/ci-performance-optimization-a161mi
Jul 14, 2026
Merged

ci: faster Test Core — affected-only PR tests, single spec suite run#2925
os-zhuang merged 1 commit into
mainfrom
claude/ci-performance-optimization-a161mi

Conversation

@os-zhuang

Copy link
Copy Markdown
Contributor

Why

Test Core is the CI critical path at ~12 min per run (turbo run test alone is 10m20s; a recent main run executed 129 tasks with only 3 turbo cache hits). Log analysis of run 29339722865 showed two structural wastes:

  1. The 250-file @objectstack/spec suite runs twice per run — once inside turbo run test (~53 s), then again in full under test:coverage (~38 s) just to collect coverage for *.zod.ts.
  2. Every PR runs the full 75-package matrix, even when the diff can't affect most of it.

What

  • PRs: turbo run test --affected (with TURBO_SCM_BASE pinned to the PR base SHA, checkout at fetch-depth: 0). Spec-touching PRs still fan out to (almost) everything since spec is the graph root; PRs that don't touch spec skip the bulk of the matrix. Verified locally: this PR's own diff resolves to 9 affected test tasks instead of 75.
  • Pushes to main: unchanged full run, except @objectstack/spec is excluded from the plain test pass — the coverage step executes the identical suite once, with coverage. Coverage generation + artifact upload are now push-only (they previously also ran on PRs).
  • Concurrency: superseded runs on the same PR/branch are cancelled (same group expression objectui already uses).
  • Empty test scripts dropped: packages/apps/{account,setup,studio} have zero test files but each booted a vitest run --passWithNoTests process every run. Verified turbo treats the missing script as <NONEXISTENT> and skips it.

Expected effect

Push runs: ~1 min faster (single spec pass + no vitest no-op boots). Non-spec PRs: several minutes faster depending on the affected subgraph. No test is deleted or weakened — PR coverage collection moves to push-only, which matches how objectui already gates Codecov.

Not in this PR (follow-ups from the CI-time evaluation)

  • Sharing bootStack across dogfood suites (44 full-stack boots dominate the 7-min Dogfood job).
  • De-duplicating the ~11 dogfood RLS fixtures vs objectstack verify --rls + unit-layer RLS tests.
  • Moving example-showcase:test (141 s, import-dominated) off the Test Core critical path.

🤖 Generated with Claude Code

https://claude.ai/code/session_0128mspMLGhBrCzww6jmVUaR


Generated by Claude Code

- Add workflow-level concurrency (cancel superseded runs), mirroring objectui.
- PRs now run 'turbo run test --affected' against the PR base instead of the
  full 75-package matrix; spec-rooted PRs still fan out to dependents.
- Push runs exclude @objectstack/spec from the plain test pass: the coverage
  step already executes the identical 250-file suite once with coverage.
  Previously every push ran the whole spec suite twice. Coverage generation
  and its artifact upload are now push-only.
- Drop no-op 'vitest run --passWithNoTests' test scripts from the three
  packages with zero test files (account, setup, studio) so turbo skips them
  instead of booting vitest for nothing.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0128mspMLGhBrCzww6jmVUaR
@vercel

vercel Bot commented Jul 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
spec Ready Ready Preview, Comment Jul 14, 2026 3:29pm

Request Review

@github-actions github-actions Bot added size/s ci/cd dependencies Pull requests that update a dependency file labels Jul 14, 2026
@github-actions

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

This PR changes 1 package(s): packages/apps.

2 hand-written doc(s) reference the affected code and may need an implementation-accuracy re-verification:

  • content/docs/concepts/north-star.mdx (via packages/apps)
  • content/docs/ui/setup-app.mdx (via packages/apps)

Advisory only. To re-verify, run the docs-accuracy-audit workflow scoped to these files:
node scripts/docs-audit/affected-docs.mjs origin/main → pass the list as args.docs.

@os-zhuang os-zhuang added skip-changeset PR has no user-facing published change; bypasses the changeset gate and removed ci/cd dependencies Pull requests that update a dependency file size/s labels Jul 14, 2026 — with Claude
@github-actions github-actions Bot added ci/cd dependencies Pull requests that update a dependency file size/s and removed size/s labels Jul 14, 2026
@os-zhuang
os-zhuang marked this pull request as ready for review July 14, 2026 15:41
@os-zhuang
os-zhuang merged commit cbf654b into main Jul 14, 2026
28 of 29 checks passed
@os-zhuang
os-zhuang deleted the claude/ci-performance-optimization-a161mi branch July 14, 2026 15:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci/cd dependencies Pull requests that update a dependency file size/s skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants